Impact factor distortions.

THIS EDITORIAL COINCIDES WITH THE RELEASE OF THE SAN FRANCISCO DECLARATION ON RESEARCH Assessment (DORA), the outcome of a gathering of concerned scientists at the December 2012 meeting of the American Society for Cell Biology.* To correct distortions in the evaluation of scientifi c research, DORA aims to stop the use of the “journal impact factor” in judging an individual scientist’s work. The Declaration states that the impact factor must not be used as “a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.” DORA also provides a list of specifi c actions, targeted at improving the way scientifi c publications are assessed, to be taken by funding agencies, institutions, publishers, researchers, and the organizations that supply metrics. These recommendations have thus far been endorsed by more than 150 leading scientists and 75 scientifi c organizations, including the American Association for the Advancement of Science (the publisher of Science). Here are some reasons why: The impact factor, a number calculated annually for each scientifi c journal based on the average number of times its articles have been referenced in other articles, was never intended to be used to evaluate individual scientists, but rather as a measure of journal quality. However, it has been increasingly misused in this way, with scientists now being ranked by weighting each of their publications according to the impact factor of the journal in which it appeared. For this reason, I have seen curricula vitae in which a scientist annotates each of his or her publications with its journal impact factor listed to three signifi cant decimal places (for example, 11.345). And in some nations, publication in a journal with an impact factor below 5.0 is offi cially of zero value. As frequently pointed out by leading scientists, this impact factor mania makes no sense.† The misuse of the journal impact factor is highly destructive, inviting a gaming of the metric that can bias journals against publishing important papers in fi elds (such as social sciences and ecology) that are much less cited than others (such as biomedicine). And it wastes the time of scientists by overloading highly cited journals such as Science with inappropriate submissions from researchers who are desperate to gain points from their evaluators.‡ But perhaps the most destructive result of any automated scoring of a researcher’s quality is the “me-too science” that it encourages. Any evaluation system in which the mere number of a researcher’s publications increases his or her score creates a strong disincentive to pursue risky and potentially groundbreaking work, because it takes years to create a new approach in a new experimental context, during which no publications should be expected. Such metrics further block innovation because they encourage scientists to work in areas of science that are already highly populated, as it is only in these fi elds that large numbers of scientists can be expected to reference one’s work, no matter how outstanding. Thus, for example, in my own fi eld of cell biology, new tools now allow powerful approaches to understanding how a large single-celled organism such as the cilate Stentor can precisely pattern its surface, creating organlike features that are presently associated only with multicellular organisms.§ The answers are likely to bring new insights into how all cells operate, including our own. But only the very bravest of young scientists can be expected to venture into such a poorly populated research area, unless automated numerical evaluations of individuals are eliminated. The DORA recommendations are critical for keeping science healthy. As a bottom line, the leaders of the scientifi c enterprise must accept full responsibility for thoughtfully analyzing the scientifi c contributions of other researchers. To do so in a meaningful way requires the actual reading of a small selected set of each researcher’s publications, a task that must not be passed by default to journal editors.